Development of entrustable professional activities for family medicine in South Africa.

South Africa is undergoing a significant shift towards implementing enhanced workplace-based assessment methodologies across various specialist training programmes, including family medicine. This paradigm involves the evaluation of Entrustable Professional Activities (EPAs) through comprehensive portfolios of evidence, which a local and national clinical competency committee then assesses. The initial phase of this transformative journey entails the meticulous development of EPAs rooted in discrete units of work. Each EPA delineates the registrar's level of entrustment for autonomous practice, along with the specific supervision requirements. This concise report details the collaborative effort within the discipline of family medicine in South Africa, culminating in the consensus formation of 22 meticulously crafted EPAs for postgraduate family medicine training. The article intricately outlines the systematic structuring and rationale behind the EPAs, elucidating the iterative process employed in their development. Notably, this marks a groundbreaking milestone as the first comprehensive documentation of EPAs nationally for family medicine training in Africa.

Competency-based and less time-bound: a new approach to the macro-structure of a medical school curriculum.

Shifting to a competency-based (CBME) and not time-bound curricular structure is challenging in the undergraduate medical education (UME) setting for a number of reasons. There are few examples of broad scale CBME-driven interventions that make the UME program less time-bound. However, given the range of student ability and varying speed of acquisition of competencies, this is an area in need of focus. This paper describes a model that uses the macro structure of a UME program to make UME curricula less time-bound, and driven more by student competency acquisition and individual student goals. The 3 + 1 curricular model was derived from the mission of the school, and includes a 3-year core curriculum that all students complete and an individualized phase. Students have an 18 month individualized educational program that meets their developmental needs and their educational and professional goals. This is achieved through a highly structured advising system, including the creation of an Individualized Learning Plan, driven by specific goals and targeted Entrustable Professional Activities (EPA). Students who struggle in achieving core competencies can use individualized time to support competency development and EPA acquisition. For students who have mastered core competencies, options include obtaining a masters degree, clinical immersion, research, and community-based experiences. Students can also graduate after the 3-year core curriculum, and enter residency one year early. Structural approaches such as this may contribute to the norming of the developmental nature of medical education, and can advance culture and systems that support CBME implementation at the UME level.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Enabling Factors for the Implementation of Competency-Based Curricula in Colleges and Schools of Pharmacy.

OBJECTIVES: To review the implementation drivers of competency-based pharmacy education (CBPE) and provide recommendations for enablers. FINDINGS: Competency-based education is an emerging model in the health professions, focusing on time-variable competency development and achievement compared with a time-bound, course-based, traditional model. CBPE is an outcomes-based organized framework of competencies enabling pharmacists to meet health care and societal needs. However, challenges need to be recognized and overcome for the successful implementation of CBPE. Competency drivers include defining the competencies and roles of stakeholders, developing transparent learning trajectories and aligned assessments, and establishing lifetime development programs for stakeholders. Organization drivers include developing support systems for stakeholders; facilitating connections between all educational experiences; and having transparent assessment plans, policies, and procedures that align with core CBPE precepts, including the sustainability of time-variability. Leadership drivers include establishing growth mindset and facilitating a culture of connection between workplace and educational environments, program advocacy by institutional leaders, accepting failures as part of the process, shifting the organizational culture away from learner differentiation toward competence, and maintaining sufficient administrative capability to support CBPE. SUMMARY: The successful implementation of CBPE involves enabling the competency, organization, and leadership drivers that will lead to program success. More research is needed in the areas of creation, implementation, and assessment of CBPE to determine success in this model. We have reviewed and provided recommendations to enable the drivers of successful implementation of CBPE.

Clinical supervisors' experience of a first-time application of entrustable professional activities in clinical supervision of medical students: findings from a Swedish pilot study.

BACKGROUND: Recently, all medical universities in Sweden jointly developed a framework for Entrustable Professional Activities (EPAs) for work-based training and assessment. This framework is now being introduced nationally in the new 6-year undergraduate medical programme that directly lead to a licence to practise. When EPAs are introduced, it is of central importance to gain clinical supervisors' acceptance to apply the framework in their supervision of students. The aim of this study was therefore to investigate how clinical supervisors, not familiar with EPAs, experience clinical supervision using the framework for EPAs. METHODS: We used a purposive sampling to recruit clinical supervisors. They were given written information on EPAs with a selection of suitable EPAs and the Swedish observation rating scale for assessment of autonomy, and they were offered to attend a 30-minute introductory web course. The participants were informed that EPAs were to be tested, and the students were asked to participate. After the study period the clinical supervisors participated in semi-structured interviews. Inductive qualitative content analysis was used to analyse the transcribed interviews. RESULTS: Three general themes emerged in the qualitative analysis: Promoting Feedback, Trusting Assessments and Engaging Stakeholders. The participants described benefits from using EPAs, but pointed out a need for preparation and adaptation to facilitate implementation. The structure was perceived to provide structured support for feedback, student involvement, entrustment decisions, enabling supervisors to allow the students to do more things independently, although some expressed caution to rely on others' assessments. Another concern was whether assessments of EPAs would be perceived as a form of examination, steeling focus from formative feedback. To understand the concept of EPA, the short web-based course and written information was regarded as sufficient. However, concern was expressed whether EPA could be applied by all clinical supervisors. Involvement and adaption of the workplace was pointed out as important since more frequent observation and feedback, with documentation requirements, increase the time required for supervision. CONCLUSIONS: EPAs were accepted as beneficial, promoting structured feedback and assessments of the students' autonomy. Preparation of supervisors and students as well as involvement and adaptation of the workplace was pointed out as important.

Trauma resident exposure in Canada and operative numbers (TraumaRECON): a national multicentre retrospective review of operative and nonoperative trauma teaching.

BACKGROUND: General surgeons play an important role in the provision of trauma care in Canada and the current extent of their trauma experience during training is unknown. We sought to quantify the operative and nonoperative educational experiences among Canadian general surgery trainees. METHODS: We conducted a multicentre retrospective study of major operative exposures experienced by general surgery residents, as identified using institutional trauma registries and subsequent chart-level review, for 2008-2018. We also conducted a site survey on trauma education and structure. RESULTS: We collected data on operative exposure for general surgery residents from 7 programs and survey data from 10 programs. Operations predominantly occurred after hours (73% after 1700 or on weekends) and general surgery residents were absent from a substantial proportion (25%) of relevant trauma operations. The structure of trauma education was heterogeneous among programs, with considerable site-specific variability in the involvement of surgical specialties in trauma care. During their training, graduating general surgery residents each experienced around 4 index trauma laparotomies, 1 splenectomy, 1 thoracotomy, and 0 neck explorations for trauma. CONCLUSION: General surgery residents who train in Canada receive variable and limited exposure to operative and nonoperative trauma care. These data can be used as a baseline to inform the application of competency-based medical education in trauma care for general surgery training in Canada.

AI in medical education: uses of AI in construction type A MCQs.

BACKGROUND: The introduction of competency-based education models, student centers, and the increased use of formative assessments have led to demands for high-quality test items to be used in assessments. This study aimed to assess the use of an AI tool to generate MCQs type A and evaluate its quality. METHODS: The study design was cross-sectional analytics conducted from June 2023 to August 2023. This study utilized formative TBL. The AI tool (ChatPdf.com) was selected to generate MCQs type A. The generated items were evaluated using a questionnaire for subject experts and an item (psychometric) analysis. The questionnaire to the subject experts about items was formed based on item quality and rating of item difficulty. RESULTS: The total number of recurrent staff members as experts was 25, and the questionnaire response rate was 68%. The quality of the items ranged from good to excellent. None of the items had scenarios or vignettes and were direct. According to the expert's rating, easy items represented 80%, and only two had moderate difficulty (20%). Only one item out of the two moderate difficulties had the same difficulty index. The total number of students participating in TBL was 48. The mean mark was 4.8 ± 1.7 out of 10. The KR20 is 0.68. Most items were of moderately difficult (90%) and only one was difficult (10%). The discrimination index of the items ranged from 0.77 to 0.15. Items with excellent discrimination represented 50% (5), items with good discrimination were 3 (30%), and only one time was poor (10%), and one was none discriminating. The non-functional distractors were 26 (86.7%), and the number of non-functional distractors was four (13.3%). According to distractor analysis, 60% of the items were excellent, and 40% were good. A significant correlation (p = 0.4, r = 0.30) was found between the difficulty and discrimination indices. CONCLUSION: Items constructed using AI had good psychometric properties and quality, measuring higher-order domains. AI allows the construction of many items within a short time. We hope this paper brings the use of AI in item generation and the associated challenges into a multi-layered discussion that will eventually lead to improvements in item generation and assessment in general.

Developing a competency framework for training with simulations in healthcare: a qualitative study.

BACKGROUND: Simulation is a technique used to create an experience without going through the real event. Competency-based medical education focuses on outcomes and ensures professionals have the necessary knowledge, skills, and attitudes. The purpose of this study was to develop a set of competencies for the instructors providing basic and advanced levels of simulation-based training in healthcare. METHODS: We conducted a qualitative study in three steps, with each next step building on and influenced by the previous one. First, we conducted a literature review, then a consensus development panel, and finally a three-step Delphi process. The participants were experts in the fields of healthcare, education, and simulations. RESULTS: The six main competencies identified for the instructor providing simulation-based training at the basic level in healthcare include knowledge of simulation training, education/training development, education/training performance, human factors, ethics in simulation, and assessment. An instructor providing simulation-based training at an advanced level in healthcare should also possess the following five competencies: policies and procedures, organisation and coordination, research, quality improvement, and crisis management. CONCLUSION: The identified competencies can serve as a valuable resource for simulation educators and organisations involved in simulation education, to plan curriculum and implement a continuous train-the-trainers programme.

Current use of simulation for EPA assessment in emergency medicine.

OBJECTIVE: Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment. METHODS: A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views. RESULTS: A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment. CONCLUSIONS: Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

RéSUMé: OBJECTIF: Il y a environ cinq ans, les programmes de médecine d'urgence du Collège royal au Canada ont mis en place un paradigme basé sur les compétences et ont introduit l'utilisation d'activités professionnelles confiables (APC) pour l'évaluation des unités d'activité professionnelle afin d'évaluer les stagiaires. De nombreux programmes d'enseignement médical basés sur les compétences (CBME) prévoient l'évaluation des compétences par l'observation des APC. Bien que les APC soient fréquemment évaluées en milieu clinique, la simulation est également utilisée. Cette étude visait à caractériser l'utilisation de la simulation pour l'évaluation de l'APC. MéTHODES: Un guide d'entretien d'étude a été élaboré conjointement par tous les auteurs de l'étude et a suivi les meilleures pratiques en matière d'élaboration d'enquêtes. Un entretien national a été réalisé avec les directeurs de programmes ou les directeurs adjoints de tous les programmes de médecine d'urgence du Collège royal au Canada. Les entretiens ont été menés sur Microsoft Teams, enregistrés et transcrits à l'aide du service de transcription de Microsoft Teams. Les transcriptions des échantillons ont été analysées pour développer des thèmes. Les thèmes ont ensuite été revus par les co-auteurs pour s'assurer qu'ils étaient représentatifs des points de vue des participants. RéSULTATS: Un taux de réponse de 64,7 % a été obtenu. La simulation a été largement adoptée par les programmes de formation en médecine d'urgence. Toutes les personnes interrogées se sont montrées favorables à l'utilisation de la simulation pour l'évaluation de l'APE pour de nombreuses raisons. Cependant, les DP ont reconnu des limites et l'analyse thématique a révélé certains thèmes et tensions liés à l'utilisation de la simulation pour l'évaluation de l'APC. L'analyse thématique a révélé six thèmes majeurs : un appui généralisé à l'utilisation de la simulation pour l'évaluation de l'APC, inquiétudes concernant la possibilité que l'évaluation de l'APC devienne un exercice de type « case à cocher ¼, des obstacles logistiques limitant l'utilisation de la simulation pour l'évaluation de l'APC, les perceptions variées quant à l'authenticité de l'utilisation de la simulation pour l'évaluation de l'APC, le potentiel de la simulation pour l'évaluation de l'APC de compromettre la sécurité psychologique des apprenants, et des suggestions pour l'optimisation de l'utilisation de la simulation pour l'évaluation de l'APC. CONCLUSIONS: Nos résultats offrent un aperçu à d'autres programmes et spécialités sur la meilleure façon d'utiliser la simulation pour l'évaluation de l'APC. Les programmes devraient utiliser ces résultats lorsqu'ils envisagent d'utiliser la simulation pour l'évaluation de l'APC.

Percepción sobre la formación en cuidados paliativos de los estudiantes de las especializaciones de Cirugía general en Colombia / Perception of palliative care training among students of General Surgery specializations in Colombia

Introducción. Los cuidados paliativos responden al sufrimiento de pacientes terminales y requieren personal entrenado para la intervención. Forman parte de la actividad en cirugía, sin embargo, no encontramos información sobre la educación de postgrado en cirugía en Colombia. El objetivo de este estudio fue evaluar el nivel de conocimientos en cuidados paliativos, la calidad de la formación y las estrategias pedagógicas en los residentes. Métodos. Estudio observacional con recolección de la información autodiligenciada por medio electrónico. Resultados. Participaron 228 residentes, 7,8 % mencionaron asistir a rotación en cuidado paliativo y 66,6 % tener contacto con especialistas en cuidados paliativos. El 30,7 % no identificó una estrategia pedagógica clara. El 29,3 % tuvo alto nivel de conocimiento y 21,1 % adecuada calidad de formación. El 83,8 % tuvo un alto nivel en el manejo de obstrucción intestinal. No hubo asociación entre el nivel de conocimiento y las variables evaluadas (p>0,05). Conclusiones. Ni el aprendizaje recibido, ni el año de entrenamiento tuvieron efecto en el nivel percibido de conocimiento. Las competencias en cuidados paliativos, sus métodos y la calidad del aprendizaje son deficientes a nivel de postgrado en cirugía en Colombia. Probablemente está en un currículo oculto. Es necesario implementar estrategias pedagógicas en los currículos de estudios de los programas de formación de cirujanos.

Introduction. Palliative care responds to the suffering of terminal patients and requires trained personnel for intervention. They are part of the activity in surgery; however, we did not find information about postgraduate education in surgery in Colombia. The objective of this study was to evaluate the level of knowledge in palliative care, the quality of training and pedagogical strategies in residents. Methods. Observational study with self-completed information collection by electronic means. Results. A total of 228 residents participated, 7.8% mentioned a palliative care rotation and 66.6% mentioned having contact with palliative care specialists; 30.7% did not identify a clear pedagogical strategy; 29.3% had a high level of knowledge and 21.1% had adequate quality of training; 83.8% had a high level in the management of intestinal obstruction. There was no association between the level of knowledge and the variables evaluated (p>0.05). Conclusions. Neither the learning received nor the year of training had an effect on the perceived level of knowledge. Competencies in palliative care, its methods and the quality of learning are deficient at the postgraduate level in surgery in Colombia. It is probably on a hidden resume. It is necessary to implement pedagogical strategies in the study curricula of surgeon training programs.

The Enneagram as a Tool for Resident Wellness and Correlation With Accreditation Council for Graduate Medical Education Milestone Achievements.

INTRODUCTION: The Enneagram is an ancient personality typing system developed to improve self-knowledge. Broken down into nine personality types, each is driven by a core motivating factor. Other personality assessments have been used to study the personality profile of surgeons. The purpose of this study is to evaluate the variability in Enneagram type among a single institution's general surgery residents. METHODS: All categorical general surgery residents at a single institution completed an online Enneagram assessment as part of a wellness initiative. Accreditation Council for Graduate Medical Education milestone levels for professionalism (PRO) and interpersonal and communication skills were collected for each resident's intern year. Milestone levels were compared between the nine Enneagram types. RESULTS: All nine Enneagram types were represented among surveyed residents. The most frequent Enneagram type was type 3 (20.69%). There was no significant difference between PRO (P = 0.322) and interpersonal and communication skills (P = 0.645) scores among residents distributed by Enneagram type. CONCLUSIONS: Regardless of core Enneagram type, general surgery residents in this study all achieved appropriate Accreditation Council for Graduate Medical Education milestone levels for entry level of training. The Enneagram can provide self-awareness and understanding of resident differences but does not impact initial assessment of competency in PRO and interpersonal communication skills.

Toma de decisiones de encomienda y evaluación de las actividades profesionales confiables en la educación médica basada en competencias: una revisión de alcance y análisis temático / Entrustment decision-making and evaluation of entrustable professional activities in the competency- based medical education: a scoping review and thematic analysis

Introducción: La evaluación de las actividades profesionales confiables implica que el tutor tome decisiones de encomienda que posibiliten el desarrollo de competencias adecuadas de los futuros especialistas. Objetivo. El objetivo es conocer los componentes que sustentan la toma de decisiones de encomienda como herramienta de evaluación efectiva, segura y eficaz de las actividades profesionales confiables en la educación médica basada en competencias. Materiales y métodos: Se realizó una revisión sistemática en las bases de datos PubMed, BIREME-BVS y Web of Science. Se procedió al análisis y síntesis de los artículos, de acuerdo con la metodología JBI para las revisiones de alcance; el análisis se profundizó en otros artículos de revistas especializadas y citas bibliográficas relacionadas; y el manuscrito final se efectuó con base en las recomendaciones PRISMA-ScR. Resultados: La toma de decisiones de encomienda se centra en la relación tutor-estudiante, con base principalmente en la proactividad, la integridad, la capacidad y la humildad del estudiante, que confluyen en la confianza del tutor para sobrepasar la ‘zona de desarrollo próximo’ y alcanzar un siguiente nivel. Aunque eso signifique un determinado riesgo inicial sobre la seguridad de la atención médica, permite gradualmente generar la autonomía del estudiante. Conclusiones: No cabe duda de que la toma de decisiones de encomienda se basa en la confianza, la encomienda, la supervisión y la autonomía. Una escala retrospectiva-prospectiva que incluya la confianza-encomienda-supervisión-autonomía permite una adecuada evaluación de las actividades profesionales confiables y, por ende, la evaluación de las competencias.(AU)

Introduction: The evaluation of entrustable professional activities implies that the tutor makes assignment decisions that enable the development of appropriate competencies of future specialists. Aim. The aim of this work is to recognize the components that support entrustment decision making as an effective, safe and efficient evaluation tool of entrustable professional activities, in competency-based medical education. Materials and methods: A systematic review was carried out in the PubMed, BIREME-BVS and Web of Science databases. The analysis and synthesis of the articles was carried out in accordance with the JBI methodology for scoping reviews; and was further conducted including other articles from specialized journals and related bibliographic citations. The final manuscript was prepared based on the PRISMA-ScR recommendations.Results: Entrustment decision-making focuses on the tutor-student relationship, based mainly on the student’s proactivity, integrity, ability and humility, which converge in the tutor’s confidence to surpass the ‘zone of proximal development’ and reach the next level. Although this means a certain initial risk to the safety of medical care, it gradually generates the student’s autonomy. Conclusions: There is no doubt that entrustment decision-making is based on trust, entrustment, supervision and autonomy. A retrospective-prospective scale that includes trust-entrustment-supervision-autonomy allows for an adequate evaluation of entrustable professional activities and, therefore, the evaluation of competencies.(AU)

Exploring the Quality of Feedback in Entrustable Professional Activity Narratives Across 24 Residency Training Programs.

Background Competency-based medical education (CBME) has been implemented in many residency training programs across Canada. A key component of CBME is documentation of frequent low-stakes workplace-based assessments to track trainee progression over time. Critically, the quality of narrative feedback is imperative for trainees to accumulate a body of evidence of their progress. Suboptimal narrative feedback will challenge accurate decision-making, such as promotion to the next stage of training. Objective To explore the quality of documented feedback provided on workplace-based assessments by examining and scoring narrative comments using a published quality scoring framework. Methods We employed a retrospective cohort secondary analysis of existing data using a sample of 25% of entrustable professional activity (EPA) observations from trainee portfolios from 24 programs in one institution in Canada from July 2019 to June 2020. Statistical analyses explore the variance of scores between programs (Kruskal-Wallis rank sum test) and potential associations between program size, CBME launch year, and medical versus surgical specialties (Spearman's rho). Results Mean quality scores of 5681 narrative comments ranged from 2.0±1.2 to 3.4±1.4 out of 5 across programs. A significant and moderate difference in the quality of feedback across programs was identified (χ2=321.38, P<.001, ε2=0.06). Smaller programs and those with an earlier launch year performed better (P<.001). No significant difference was found in quality score when comparing surgical/procedural and medical programs that transitioned to CBME in this institution (P=.65). Conclusions This study illustrates the complexity of examining the quality of narrative comments provided to trainees through EPA assessments.

Exploring the current state of clinical and practical teaching in obstetrics and gynecology in the era of competency-based education: a nationwide survey among German teaching coordinators.

BACKGROUND: Obstetrics and gynecology (OB/GYN) is an essential medical field that focuses on women's health. Universities aim to provide high-quality healthcare services to women through comprehensive education of medical students. In Germany, medical education is undergoing a phase of restructuring towards the implementation of competency-based learning. The objective of the current survey was to gain insights into the teaching methods, resources, and challenges at German medical universities in the field OB/GYN. This aims to document the current state of medical education and derive potential suggestions for improvements in the era of competency-based learning. The survey was conducted with teaching coordinators from the majority of OB/GYN departments at German universities. METHODS: A questionnaire was sent to the teaching coordinators in all 41 OB/GYN departments at German university hospitals. The survey was delivered via email with a link to an online survey platform. RESULTS: The study received 30 responses from 41 universities. Differences were observed in the work environment of teaching coordinators concerning release from clinical duties for teaching purposes and specialized academic training. Overall, medical education and student motivation were perceived positively, with noticeable gaps, particularly in practical gynecological training. Deficiencies in supervision and feedback mechanisms were also evident. Subfields such as urogynecology and reproductive medicine appear to be underrepresented in the curriculum, correlating with poorer student performance. E-learning was widely utilized and considered advantageous. CONCLUSION: The present study provides valuable insights into the current state of medical education in OB/GYN at German universities from the perspective of teaching experts. We highlight current deficits, discuss approaches to overcome present obstacles, and provide suggestions for improvement.

Competency-Based Medical Education at Scale: A Road Map for Transforming National Systems of Postgraduate Medical Education.

In the past decade, the Canadian system of postgraduate medical education has been transformed with the implementation of a new approach to competency based medical education called Competence by Design. The Royal College of Physicians and Surgeons of Canada (Royal College) developed an approach to time-variable competency based medical education and adapted that design for medical, surgical, and diagnostic disciplines. New educational standards and entrustable professional activities consistent with this approach were co-created with 67 specialties and subspecialties, and implementation was scaled up across 17 universities and over 1000 postgraduate training programs. Partner engagement, systematic design of workshops to create discipline specific competency-based standards of education, and agile adaptation were all key ingredients for success. This paper describes the strategies applied by the Royal College, lessons learned regarding transformative change in the complex system of postgraduate medical education, and the current status of the Competence by Design initiative. The approach taken and lessons learned by the Royal College may be useful for other educators who are planning a transformation to CBME or any other major educational reform.

Implementing Competence Committees on a National Scale: Design and Lessons Learned.

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.

Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Enabling Implementation of Competency Based Medical Education through an Outcomes-Focused Accreditation System.

Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.